Uploaded image for project: 'Apache Storm'
  1. Apache Storm
  2. STORM-2689

storm-kafka-examples and storm-kafka-client-examples are difficult for new users to run

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.0.0, 1.1.1
    • 2.0.0
    • examples
    • None

    Description

      The storm-kafka-examples and storm-kafka-client-examples projects configure their dependencies in a way that makes them difficult to run for a new user. The other example projects set up a provided dependency on storm-client, and otherwise include all their dependencies in a shaded jar.

      storm-kafka(-client) by default produce jars without several necessary dependencies, e.g. the Kafka client libraries. The provided.scope Maven parameter was intended to be used to allow users to produce a shaded jar with all dependencies, but if provided scope is set to compile, the resulting jar will also contain storm-client. This prevents the jar from running on a real cluster.

      While users can work around this by producing the slim jar and using --artifacts when submitting the topology, this is unnecessarily tedious. We should just produce a fat jar by default, then mention in the example documentation that --artifacts is there for users that want to make slimmer jars.

      Edit:
      This issue now includes simplifying storm-kafka-examples and storm-kafka-client-examples in general. The examples demonstrate use of State and DRPC when the focus should be on how to use storm-kafka(-client). It also causes the modules to have some undesirable dependencies, e.g. they both depend on storm-starter.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            srdo Stig Rohde Døssing
            srdo Stig Rohde Døssing
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 2h 20m
                2h 20m

                Slack

                  Issue deployment